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Overview ' • 

■\ . . * — ;v " - , * •/*..< 

\ .■ " ■ ' " ' : -■■ . < . . 

\ '. ' • '■ - * ? • . ' ' : . : ■ ' . 

The Education Consolidation and Improvement Act (EG IA) of L 

1981 wis designed in £>art to change the nature of educational 

evalWatioh x required by the federal government at state and local 

levels* Will the new requirements indeed consolidate and jpaprove 

evaluation activities? Because* this new legislation will 

probably have t^e greatest impact on the nature of evaluation 

practice In education since the passage of the Elementary and 

Secondary Education Act of 19165, we conducted" a ten-month study 

of five atate departments of education and five large school * 

districts to learn how evaluation, units responded to thip 

legislation during 1982* - 

One of the basic moves by the Reagan Administration in its 

effort to change the federal role in education was to propose the 

consolidation of most categorical education programs into* a few 

large block grants. Congress, reacting to pressure from those 

representing the disadvantaged , the handicapped* "and others, 

resisted this move. '.. " ' - * 

In the end, the major legislation for the disadvantaged 

(Title I of the Elementary and Secondary Education Act, ESEA) , 

the handicapped (PL 94^14,3, Education for all Handicapped 

Children tat), voca^iQnal and adult education (ThiT Vocational 

Education Act), and a few other programs (e.g., impact aid, ^ 

student financial assistance) # survived as separate entities. 

But, .30-4 0 previous categorical aid programs were combined into 

one /arge block grant* Included were such programs as basic 

^skills education, metric education, consumer education, gifted 

•and talented education, and programs provid4jjg_funds for library 

resources, textbooks and instructional equipment, guidance and 

counseling, desegregation, and teacher training and: in4ser vice* : 

* The disadvantaged student legislation, Title and\ the block 

grant legislation ' were Combined in Subtitle D of Ti.Mle* V of the 

Omnibus .Budget 'Mbonciliatlton Act of 1981 (PL 97-35) as 

Chapters 1 and 2, respectively,,^ of the Bducatio^n Gonsol Ida t ion 



and Improvement Act of if 1 81 (ECIA) - • The resultant effects of 



Oiapter 1. and Qiapter 2 legislation on evaluation practice offer ; 
graphic examples of the mixed results achieved . the '< ^ 

Adrofhistration,, • 1 ; \ • f 

The reasoning behind Chapter 1 and\Chapter 2 is A ref lecteg in • 

■ • . ■ ■ •> ; ■. v * . ; . v , , i V ■ ■ ■ « v 

the legislation itself (PL 97-35, $Ogust ,13, 1982} , The ..' ■". ' 
Chapter 1 Declaration of Policy state:; that it in the intent of 

7 \ 7 \ ..; ..• . ••: • 7 \ . 7 ... , \,-, s ... 

the legislation to: . . 



eliminate burdensome, unnecessary* t and unproductive papes 
work and free* the schooisf of unnecessary . F$0&fmX 
supervision, direction, and control * , 



iupc 

k (p. 95 STAT, 464) *^"\, 

In addition, it "is intended that the legislationVwill do away with 

overly prescriptive regulations and administrative, ^ ~ - 
burdens which ^re not necessary for fiscal accountability , 
and make no contribution to the instructional program, 

* * • ., • (p. iS a STAT- 464) 

"The purpose of i:he block grant (Chapter 2) was essentially to 
free state and local education agencies, to s^et tjieir own 
priorities and conduct programs as they srfw fit. Beyond 't^is 
intent, the new legislation was: , ■-, ^ ■ * 

to do so in a manner designed to greatly reduce the 

enormous aflministrative^and paperwork burden imposed on 

" schools at the .expense of their ability teb* educate 

children. * * ■ - 

1 . * * (p. 95 STAT, «|y 

Clearly, the intent of EC1A was to ahif t the responsibility 
for setting policy, monitoring fiscal procedures, and^ designing , 
operating, and evaluating educational programs out of J 
Washington, D.C, f? The full impact of this shift on /state 
education agencies (SEAs) and local education agencies s (l^As) and 
ultimately on this nation's children is now only beginning to 
manifest itself. To date, the JECI A legislation does not appear 
^to have entirely had the desired effect, 

Donald Reagan erftered the White House tw« years ago with 
1 a commitment to overhaul the federal role in American 
education. 
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"Education, B he toldNthtf: country in a televised address,- 
"is -the principal responsibility tof local school systems, 
teacher parents, "citizen boards and state 
governments . " . 

Two* years latmE? as* mimberk of the Administration are the 
; -JEirsi > to acknowledge , e£for j t& to redefine and reshape the 
Federal role in education shoW mixed results. , ; 

(£he New Yopk Times f November 14 ? 1982) 

1 This 'summary describes the ptfrf?oste, design, results, and 
iplieati^ns oi^m preliminary f ie^d^investigation^of ECIA impaet. 



Study Purpose 



.The study reposed here focused on the initial impact of * 
CIA. In f some case^ %these findings will hold trV^e over the; long 



Chapter „2 requirements, 'the full impact of the legislation has 



run, whiif in, other case's, add it iortaX changes ma J- alter 
dramatically- the perceptions! and impleroe n ta t ion of Chapter 1 or 
Aer\2 rec^ui 

not yet been ffelt, but these "early returns" are important; they 

- . - \ * . \ ■ . \ • 

give us insights into the initial responses to the ECIA arid help 
shape further research on this/ legislation, including longer term 
impact' studies. More specifically, the study was intended to 
illuminate, the effect or" Chapters" 1 arid 2 oh program evaluation 

operations and' on evaluation training and technical assistance 

\ - 
needs. 



Study Design 



,Five western states, namely California, Montana, Oregon, 
Utah, and Washington, were 'chosen to represent states in the 
Northwest part of the country. They therefore provide a limited 
sample of perspectives on the 'impact of ec^ia. 

Within these states, . people were interviewed wjio were (a) « 
responsible for evaluation in general and /or (b) responsible for 
Chapter 1 and Chapter 2 evaluation in particular, at the state 



.^yv-- T i the largest school district in each state. As 

er^onnel changes at state and local levels. Me 
• -±fi i over 3b professional evaluators and educators during 
>r •■£!. cu ; /, . In the November 1982 f in^l interview we tgtkad /with 

' *i / '• ■ .. •' " ' v * 'V 

"V Indents. We classified their positions as: t • 

u 5 SEA evaluation directors 1 I. 
5 LEA evaluation directors , 
5 SEA Chapter 1 evaluatoips ■ 
•V 5 jypA .Chapter 1 eyaluatdrs 

4 L%A^ Chapter sources v i- 

.. / ■ ^. jr -, ; 

A semi-structured^ interview instrument was developed- and 
revised on the basis of pilot trials. The form was modified as 
necessary to reflect changing field conditions over the course of 
the study, but the 1 same basic questions were retained throughout 
the study and they form the basis of the'tfesults presented here* 
Respondents were interviewed periodicaiCly between, Februaj^ 1982, 
and November 1982, giving a ten-month perspective on charging 
responses to^ the new legislation* , \ * 



FrelimjLnary Results 

t> ■ . 

The following summaries give a brief overview of the 
* * ■ i . ■ . ■ 

preliminary study findings and are based on the study resilts 

appearing in the complete ^report ( A Study in CdnAr asts. Effects 



of the Education Consolidation and -improvement Act of 1981 on SEA 



and LEA Evaluation * Gray, Smith, Ci/ulley, 1982, A copy o£ the 
full report is Available -from the first author.) - The fulll 
report focuses on the sometimes subtle" differenpes b^tweeni state 
and local perceptions* Provided in the bomplete report are 
examples of the differing perceptions of some of th^ more.dompleK 
concepts investigated! unnecessary and .burdensome paperwork! 
^supervision, monitoring and evaluation! flexibility "and * ^ 

decentralization; administrative burden/ The reader is therefore 
* encouraged to iread this overview in light of its purpose as a , : 
brief summary of more detailed findings* 



A 



. i Summary of Preliminary Chapter 1 Findings * 

. '/ loth state /(SEA) an4 local (LEA)" evaluators* reported that 

Title I paperwork^ was not seen as burdensome and unnecessary, nor 
f • . was any major change in this regard, reported at the local level 

relative to Chapter 1 paperwork. In addition* overall* statie^and 
local Chapter- 1 directors reported little perceived change in ^ 
* administrative or evaluation flexibility,, decentralization of 
responsibility, or administrative burden regarding rules and j 
regulations. ^A' major reason for trie, perceived continuity in 
administrative practice is that, the formal, Title I Evaluation and 
Reporting System (TIERS) is being retailed in all states ^ 
represented in this study. The changes "that have occurred 
concern dropping the collection of certain types of data 
r (e.g". f ethnic data, parent council data, staff t training data, and 
project .data) t a point specifically addressed in**the " new . 
legislation. * 

I*E^ Chapter 1 directors did, however, report a perceived 
relaxation in the testing and reporting requirements of Chapter 
1. For example one district is contemplating using a criterion 
referenced testing program. Previously, such .programs were, not 
encouraged since it is difficult to generate student pre— and 
^ post-test data which can be relied upon "to give ai accurate a 
\ picture of growth as scores from standardized, horm^referenoed 
tests. In another district, it, was said that percentile scores 
would be emphasized instead of the Normal Curve Equivalent (NCE) 
scores favored by TIERS, Both of these changes were perceived by 
local directors as effSrts to make^\esting and test results more 
meaningful to local audiences. „ ' - , ^ 

An increasedLemphasi'S on sustained effects studies was 
reported at both the ^tate and local levels, consistent with the 
^ emphasis in the legislation on such studies. Sustained effects 
^studies *trace the impact of 9 a program over time. The ^impact may 
be defined in terms of student achievement or some other - program 
characteristic of interest. Such studies can t^e very useful in 
monitoring program quality, planning program improvement, and 
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reporting program impact*, Sustained effects studies represent 
another way of making evaluations relevant to local audiences 
since they focus on questions of interest, to local decision - ^ 
makers, including administrators, teachers, parents, , and students 

Parent involvement in Chaptet' 1 programs is taking on new 
\ forms consistent with the change in emphasis in the legislation. 
For a example, in. one school district "willingness tp work™ was 
being valued over "representativeness* 'i^j forming the school 
parent councils- In another district the new Chapter 2 advisory 
council will include parents from the Chapter 1 council resulting 
in more efficient administration of the two programs- .. ' * 

A clear message- regarding Chapter ,1 came from our 
respondents,* As. a continuation of a well institutifcnaiiWed 
program, Chapter 1/Ti tie I evalua tor s will experience few 
immediate changes. ,Hbwever> some of the testing and administra- 
tive changes could aCcuiftulate po produce an impact in the future, 
possibly , resulting in less nationwide data available for Congress 
'to examine the overall effects of /chapter 1- 

Chapter 1 budget reductions have had a more dramatic impact 

en the local level than the new legislation, resulting in cuts in 

Instructional staff, and consequently , a reduction^ in the number 

of students ^served. In discussing the impact of loping almost 

one million dollars of Chapter 1 monies, a school district 

evalua tor noted, 

* v- ; 
The rear change has borne about because of decreased 

funding, which has severely crippled our program. I was 

at a school^ this morning that had five aides and a' 

teacher* last yeari this year it has one aide and a 

* half-time teacher. That's a big difference in numbers of 

kids they can serve, etc. They've reduced their math 

services to working^ with 10 kids as opposed to 40. That -4 > 

resulted Because of budgetary cuts* • not Chapter 1 

regulation. 

^fhis district budget reduction had lowered by 1000 the number of 
children served, As the respondent noted, "when you go to 
individual schools ^you really see what that means." 



The most notable immediate change regarding Chapter 1, 4 then, 
is in regard to the level of service being offered, The 
reduction in staff, in the number of students served, and perhaps 
in the quality of that service; will have long range effects on 
Oiapter 1 impact and on the overall effectiveness of compensatory 
education. * \ 



I Chapter 1 Findings s. While the ECIA legislation has 
not significantly eliminated paperwork or administra- 
tive load, these were not especially burdensome to 
jjegin with. Some changes are occurring in loosening 
testing requirements, increasing the interest in 
sustained effects, and altering parent involvement, 
changes which are likely to increase the local 
utility of Oiapter 1 evaluation activities. The more 
significant influence on Chapter 1 than the new 
legislation is the decrease in budgets which is 
dramatically reducing the number of children that can 
be served , 



Sungnary of Preliminary Chapter 2 Findings ^ 

Most states reported that their Chapter 2 evaluations will 
consist of monitoring Chapter 2 expenditures fpr materials in 
terms of simple dost accounting procedures. At least initially, 
these states will evaluate expenditures for 4 programs in terras of 
the number of students served afod staff involved, This concern 
with the numbers involved in programs is to a great extent a 
reflection of the fact that, typically, 80 percent or more of 
Chapter 2 funds will go for instructional materials and equipment 
(mostly microcomputer hardware) , The existing categorical 
programs are likely to take the remaining Chapter, 2- funds. 

K SEA respondents were well aware of the legislative mandate to 
conduct Chapter 2 evaluation studies beginning- in. fiscal year 
»L984, and are communicating this to local administrators, x Sdme 
LEA evaluation directors are considering innovative research and 
evaluation activities under Chapter 2 including,! ' ' - 

- the study of school practices and learning using 
ethnographic approaches, 

V 



the study 6f teaoriing^ehaviors and outcomes, using 
causal modeling methods, 
••«•,'= ,f< * 

- the adaptation of the evaluation components of 

previous categorical programs, ' 

. the development of locally, run mini-grant programs 

Similar non-traditional efforts may erafrge in other districts- 

Most large school districts in our study reported a reduction 
in funds under Chapter 2, in -comparison with the funds they 
received under the. previous categorical aid programs. As a 
result, large school district respondents repor ted. the_ need to 

find ways to select among programs competing for reduced funds. 

« ' y " ■ 

Some small school districts have had modest increases in funding 
'as a result of Chapter 2 allocations^ typically increases of $800 
to $2500 per district, and are seeking ways to, join- with other 
districts to gejt the most from their increases. 

Almost all the ffiA and IMK Chapter 2 respondents reported 
increased flexibility and decentralization at the local level as 
a result of the Chapter 9 2 legislation, Thmy had mixed feelings t 
about possible reductions in administrative / burden, however. 
There is also considerable uncertainty about future evaluation 
activities at the local level under Chapter 2. Once^ programming 
decisions have been made and the first round of materials and 
microcomputer equipment purchases completed, it is expected that 
evaluators will move from monitoring expenditures to assessing 
the quality and impact of Chapter 2 programs, 



\ 



Chapter 2 Findings i The ECIA legislation has 
increased flexibility and decentralization at the 
local level while reallocating some of the previous 
categorical funds to smaller districts, Most of the 
Chapter 2 funds are being spent 'on materials and " ' 
microcomputer equipment purchases, with evaluators 
primarily monitoring the flow and expenditure of 
funds, Increasingly, however, LEA evaluators will 
be doing more to evaluate the impact of the n%w 
programs at the local level. 
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Conclusions, ( . " " - 

\ . .. s . . . r . - . 

^esi early returns indicate that the EC I A legislation has 

not substantially reduced the paperwork of administrative burden 

under Chapter l r although some changes are taking place to 

Increase the local utility of evaluation efforts* ; The largest 

'changes in Chapter 1 ^have resulted from major budget cuts and 

subsequent reductions in the numbers of children- served. 

Most Oiapter 2r monies are beihg spent on materials and* 
microcomputers* The^ECIA legislation has given more money to 
smaller districts andT has increased local flexibility an4 
control* ■ Evaluators, who are now primarily monitoring the 
expenditure of funds, are expected to conduct more studies of 
program effectiveness in the next few years* 

While these results are based on only a preliminary study of 
ECIA impact in five northwestern states* they are suggestive of 
the earl^ impact of the jew Reagan legislation ! on SEA and LEA 
evaluation practice*. We may speculate that in the near future 
evaluators will be dealing with concerns such as the following at 
both the programmatic and policy levels: 



m and evaluation^ budgets^ 



© with reduced prograi 

J ^ 

- evaluators will be called upon tc. perform more 
/ cpsfc^related studies 

- evaluators will have greater difficulty meeting the 
g demand for evaluation services 

9 . \ " * 

- evaluators will need to attend more to helping^ 
program staff maintain program quality control 

m with the increasing use of microcomputer mg 

- evaluators wijLl be called upon to assess the # 
instructional impact of microcomputers ^) 

- ! ©valuators will' need to learn more themselves about 
the opdbation of microcomputers, possibly beginning 
to use them as an evaluation tool* 



".>.'''. . . ' w t ■' • .■ - i. ' 

- *© with increased LEA flexibility and control 

. • ••• * ■ * « - - . * * 8 l ' 

- e valuators may have more dif f ictulty maintaining 

evrdence of state anpi federal aecoiyitability • 

* - evaluates s will be called upon more to assess 
% v " Chapter, 1 sustained- effects ; ' " 

- u fc evpluators will* be Called upon more to help ^rail 
LEAs with' Chapter 2 evaluation 1 activities. t 

■ *" * - * ' « 

These and other, concerns will* continue 'to surface as LEAs and 

SEAs gairi^oontinued .experience under, the new ECIA legislation* 

We hope our early returns heljp fuwfe researchers sharpen their 

studies of this major federal change in educational evaluation 

policy, * e, / ^ , 
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